Virtualization system and method for hosting applications

ABSTRACT

A computer-implemented system for providing a virtualized application hosting environment comprising: a plurality of application/game servers each having a memory for storing application program code and data and a processor for executing the application program code and processing the data on behalf of a client; a virtualization service executed on each of the application/game servers, the virtualization service comprising a file redirection module for redirecting file system operations generated by the application program code to one or more virtualized file locations; and a registry redirection module for intercepting registry operations generated by the application program code and executing the registry operations on behalf of the application program code at one or more virtualized registry locations.

RELATED APPLICATION

This application is a continuation-in-part of U.S. patent applicationSer. No. 12/538,077, filed Aug. 7, 2009, entitled SYSTEM AND METHOD FORACCELERATED MACHINE SWITCHING, which claims priority to U.S. ProvisionalApplication Ser. No. 61/210,888, filed, Mar. 23, 2009, and is acontinuation-in-part (CIP) application of Ser. No. 10/315,460 filed Dec.10, 2002 entitled, “APPARATUS AND METHOD FOR WIRELESS VIDEO GAMING”,which is assigned to the assignee of the present CIP application.

TECHNICAL FIELD

The present disclosure relates generally to the field of data processingsystems and particularly to a virtualization and encryption system andmethod for hosting applications.

BACKGROUND

“Virtualization” generally refers to the use of a virtual (rather thanactual) version of something, such as a hardware platform, operatingsystem, a storage device or network resources. Platform virtualizationrelies on the creation of a “virtual machines” which behave like a realcomputer with an operating system. Software executed on these virtualmachines are separated from the underlying hardware resources. Forexample, a computer that is running Mac OS X may host a virtual machinesuch as Parallels™ or VMWare Fusion™ that looks like a computer to theWindows operating system. Once installed in Mac OS X, Windows-basedapplications can be run on that virtual machine.

Most Windows-based applications must be installed before execution. Theinstallation process copies application binaries and data files to somefixed locations in Windows, updates the registry keys and values forthat application and establishes run-time dependencies on assemblies andsystem components. In addition, an application execution results ingeneration of new files and run time modification of existing files inWindows. Consequently, a virtual machine implementation for Windowsrequires a complex set of program code and consumes significantprocessing resources when translating between Windows-based modules andnative software modules. Running such a virtualized environment on ahosting service for multiple online users, tracking file and registryupdates and providing sufficient security in an efficient,cost-effective manner is a difficult task.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure will be understood more fully from the detaileddescription that follows and from the accompanying drawings, whichhowever, should not be taken to limit the disclosed subject matter tothe specific embodiments shown, but are for explanation andunderstanding only.

FIG. 1 illustrates a system architecture for executing online videogames according to one embodiment of the invention.

FIG. 2 illustrates different communication channels over which an onlinevideo game may be played in accordance with one embodiment of theinvention.

FIG. 3 illustrates one embodiment of a system architecture forcompressing audio/video generated by a video game.

FIG. 4 illustrates a system architecture according to one embodiment ofthe invention.

FIG. 5 illustrates one embodiment of a method for installing anapplication such as a video game in a virtualized environment.

FIG. 6 illustrates one embodiment of a method for executing file systemoperations.

FIG. 7 illustrates one embodiment of a method for executing registryoperations in a virtualized environment.

FIG. 8 illustrates one embodiment of a method for encrypting applicationfiles such as video game files for an application hosting service.

FIG. 9 illustrates another embodiment of a method for securely passingencryption keys for decrypting application files such as video gamefiles for an application hosting service.

FIG. 10 illustrates one embodiment of a method for securely executingapplication program code on a hosting service by tracking processes andrelationships between processes.

DESCRIPTION OF EXAMPLE EMBODIMENTS

In the following description specific details are set forth, such asdevice types, system configurations, communication methods, etc., inorder to provide a thorough understanding of the present disclosure.However, persons having ordinary skill in the relevant arts willappreciate that these specific details may not be needed to practice theembodiments described.

The assignee of the present application has developed an online videogaming and application hosting system. Certain embodiments of thissystem are described, for example, in U.S. patent application Ser. No.12/538,077, filed Aug. 7, 2009, entitled SYSTEM AND METHOD FORACCELERATED MACHINE SWITCHING (hereinafter '077 application) whichclaims priority to U.S. Provisional Application Ser. No. 61/210,888,filed, Mar. 23, 2009, and is a continuation-in-part (CIP) application ofSer. No. 10/315,460 filed Dec. 10, 2002 entitled, “APPARATUS AND METHODFOR WIRELESS VIDEO GAMING”, which is assigned to the assignee of thepresent CIP application. These applications are sometimes referred to asthe “co-pending applications” and are incorporated herein by reference.A brief description of certain pertinent aspects of the online videogame and application hosting system described in the co-pendingapplications will now be provided, following by a detailed descriptionof a virtualization and encryption system and method for hostingapplications.

An Exemplary Online Video Game and Application Hosting System

FIG. 1 illustrates one embodiment of a video game/application hostingservice 210 described in the co-pending applications. The HostingService 210 hosts applications running on Servers 102, that accept inputfrom an Input device 121, received by Home or Office Client 115, andsent through the Internet 110 to Hosting Service 210. The Servers 102are responsive to the input, and update their video and audio outputaccordingly which is compressed through Low-Latency Video Compression104. The compressed video is then streamed through the Internet 110 tobe decompressed by the Home or Office Client 115, and then displayed onMonitor or SD/HDTV 122. This system is an low-latency streaminginteractive video system as more thoroughly described in theaforementioned “co-pending applications.”

As shown in FIG. 2, the network connection between the Hosting Service210 Home and Office Client 215 may be implemented through a wide rangeof network technologies, of varying degrees of reliability, such aswired or optical fiber technologies that are typically more reliable andwireless technologies that may be subject to unpredictable interferenceor range limitations (e.g. Wi-Fi) and are typically less reliable. Anyof these client devices may have their own user input devices (e.g.,keyboards, buttons, touch screens, track pads or inertial-sensing wands,video capture cameras and/or motion-tracking cameras, etc.), or they mayuse external input devices 221 (e.g., keyboards, mice, game controllers,inertial sensing wand, video capture cameras and/or motion trackingcameras, etc.), connected with wires or wirelessly. As described ingreater detail below, the hosting service 210 includes servers ofvarious levels of performance, including those with high-powered CPU/GPUprocessing capabilities. During playing of a game or use of anapplication on the hosting service 210, a home or office client 215receives keyboard and/or controller input from the user, and then ittransmits the controller input through the Internet 110 to the hostingservice 210 that executes the gaming program code in response andgenerates successive frames of video output (a sequence of video images)for the game or application software (e.g., if the user presses a buttonwhich would direct a character on the screen to move to the right, thegame program would then create a sequence of video images showing thecharacter moving to the right). This sequence of video images is thencompressed using a low-latency video compressor, and the hosting service210 then transmits the low-latency video stream through the Internet110. The home or office client device 215 then decodes the compressedvideo stream and renders the decompressed video images on a monitor orTV 222. Consequently, the computing and graphical hardware requirementsof the client 215 are significantly reduced. The client 215 only needsto have the processing power to forward the keyboard/controller input tothe Internet 110 and decode and decompress a compressed video streamreceived from the Internet 110, which virtually any personal computer iscapable of doing today in software on its CPU (e.g., a Intel CorporationCore Duo CPU running at approximately 2 GHz is capable of decompressing720p HDTV encoded using compressors such as H.264 and Windows MediaVC9). And, in the case of any client devices, dedicated chips can alsoperform video decompression for such standards in real-time at far lowercost and with far less power consumption than a general-purpose CPU suchas would be required for a modern PC. Notably, to perform the functionof forwarding controller input and decompressing video, home clientdevices 215 do not require any specialized graphics processing units(GPUs), optical drive or hard drives.

As games and applications software become more complex and morephoto-realistic, they will require higher-performance CPUs, GPUs, moreRAM, and larger and faster disk drives, and the computing power at thehosting service 210 may be continually upgraded, but the end user willnot be required to update the home or office client platform 215 sinceits processing requirements will remain constant for a displayresolution and frame rate with a given video decompression algorithm.Thus, the hardware limitations and compatibility issues seen today donot exist in the system illustrated in FIGS. 1-3.

Further, because the game and application software executes only inservers in the hosting service 210, there never is a copy of the game orapplication software (either in the form of optical media, or asdownloaded software) in the user's home or office (“office” as usedherein unless otherwise qualified shall include any non-residentialsetting, including, schoolrooms, for example). This significantlymitigates the likelihood of a game or application software beingillegally copied (pirated), as well as mitigating the likelihood of avaluable database that might be use by a game or applications softwarebeing pirated. Indeed, if specialized servers are required (e.g.,requiring very expensive, large or noisy equipment) to play the game orapplication software that are not practical for home or office use, theneven if a pirated copy of the game or application software wereobtained, it would not be operable in the home or office.

In one embodiment, the hosting service 210 provides software developmenttools to the game or application software developers (which refersgenerally to software development companies, game or movie studios, orgame or applications software publishers) which design video games sothat they may design games capable of being executed on the hostingservice 210. Such tools allow developers to exploit features of thehosting service that would not normally be available in a standalone PCor game console (e.g., fast access to very large databases of complexgeometry (“geometry” unless otherwise qualified shall be used herein torefer to polygons, textures, rigging, lighting, behaviors and othercomponents and parameters that define 3D datasets)).

FIG. 3 illustrates an embodiment of components of a server center forhosting service 210 utilized in the following feature descriptions. Aswith the hosting service 210 illustrated in FIG. 2, the components ofthis server center are controlled and coordinated by a hosting service210 control system unless otherwise qualified.

Inbound internet traffic 301 from user clients 215 is directed toinbound routing 302. Typically, inbound internet traffic 301 will enterthe server center via a high-speed fiber optic connection to theInternet, but any network connection means of adequate bandwidth,reliability and low latency will suffice. Inbound routing 302 is asystem of network (the network can be implemented as an Ethernetnetwork, a fiber channel network, or through any other transport means)switches and routing servers supporting the switches which takes thearriving packets and routes each packet to the appropriateapplication/game (“app/game”) server 321-322, 325. In one embodiment, apacket which is delivered to a particular app/game server represents asubset of the data received from the client and/or may betranslated/changed by other components (e.g., networking components suchas gateways and routers) within the data center. In some cases, packetswill be routed to more than one server 321-322, 325 at a time, forexample, if a game or application is running on multiple servers at oncein parallel. RAID arrays 311-312 are connected to the inbound routingnetwork 302, such that the app/game servers 321-322, 325 can read andwrite to the RAID arrays 311-312. Further, arrays 311-312 (which may beimplemented as multiple RAID arrays) is also connected to the inboundrouting 302 and data from RAID arrays 311-312 can be read from app/gameservers 321-322, 325. The inbound routing 302 may be implemented in awide range of prior art network architectures, including a treestructure of switches, with the inbound internet traffic 301 at itsroot; in a mesh structure interconnecting all of the various devices; oras an interconnected series of subnets, with concentrated trafficamongst intercommunicating device segregated from concentrated trafficamongst other devices. One type of network configuration is a SAN which,although typically used for storage devices, it can also be used forgeneral high-speed data transfer among devices. Also, the app/gameservers 321-322, 325 may each have multiple network connections to theinbound routing 302. For example, a server 321-322, 325 may have anetwork connection to a subnet attached to RAID Arrays 311-312 andanother network connection to a subnet attached to other devices.

The app/game servers 321-322, 325 may all be configured the same, somedifferently, or all differently, as previously described in relation toservers 102 in the embodiment illustrated in FIG. 1. In one embodiment,each user, when using the hosting service is typically using at leastone app/game server 321-322, 325. For the sake of simplicity ofexplanation, we shall assume a given user is using app/game server 321,but multiple servers could be used by one user, and multiple users couldshare a single app/game server 321-322, 325. The user's control input,sent from client 215 as previously described is received as inboundInternet traffic 301, and is routed through inbound routing 302 toapp/game server 321. App/game server 321 uses the user's control inputas control input to the game or application running on the server, andcomputes the next frame of video and the audio associated with it.App/game server 321 then outputs the uncompressed video/audio 329 toshared video compression 330. App/game server may output theuncompressed video via any means, including one or more Gigabit Ethernetconnections, but in one embodiment the video is output via a DVIconnection and the audio and other compression and communication channelstate information is output via a Universal Serial Bus (USB) connection.

The shared video compression 330 compresses the uncompressed video andaudio 329 from the app/game servers 321-322, 325. The compression maybeimplemented entirely in hardware, or in hardware running software. Theremay a dedicated compressor for each app/game server 321-322, 325, or ifthe compressors are fast enough, a given compressor can be used tocompress the video/audio from more than one app/game server 321-322,325. For example, at 60 fps a video frame time is 16.67 ms. If acompressor is able to compress a frame in 1 ms, then that compressorcould be used to compress the video/audio from as many as 16 app/gameservers 321-322, 325 by taking input from one server after another, withthe compressor saving the state of each video/audio compression processand switching context as it cycles amongst the video/audio streams fromthe servers. This results in substantial cost savings in compressionhardware. Since different servers will be completing frames at differenttimes, in one embodiment, the compressor resources are in a sharedhardware compression 330 with shared storage means (e.g., RAM, Flash)for storing the state of each compression process, and when a server321-322, 325 frame is complete and ready to be compressed, a controlmeans determines which compression resource is available at that time,provides the compression resource with the state of the server'scompression process and the frame of uncompressed video/audio tocompress.

Note that part of the state for each server's compression processincludes information about the compression itself, such as the previousframe's decompressed frame buffer data which may be used as a referencefor P tiles, the resolution of the video output; the quality of thecompression; the tiling structure; the allocation of bits per tiles; thecompression quality, the audio format (e.g., stereo, surround sound,Dolby® AC-3). But the compression process state also includescommunication channel state information regarding the peak data rate andwhether a previous frame is currently being output (and as result thecurrent frame should be ignored), and potentially whether there arechannel characteristics which should be considered in the compression,such as excessive packet loss, which affect decisions for thecompression (e.g., in terms of the frequency of I tiles, etc). As thepeak data rate or other channel characteristics change over time, asdetermined by an app/game server 321-322, 325 supporting each usermonitoring data sent from the client 215, the app/game server 321-322,325 sends the relevant information to the shared hardware compression330.

The shared hardware compression 330 also packetizes the compressedvideo/audio using means such as those previously described, and ifappropriate, applying FEC codes, duplicating certain data, or takingother steps to as to adequately ensure the ability of the video/audiodata stream to be received by the client 215 and decompressed with ashigh a quality and reliability as feasible.

Some applications, such as those described below, require thevideo/audio output of a given app/game server 321-322, 325 to beavailable at multiple resolutions (or in other multiple formats)simultaneously. If the app/game server 321-322, 325 so notifies theshared hardware compression 330 resource, then the uncompressed videoaudio 329 of that app/game server 321-322, 325 will be simultaneouslycompressed in different formats, different resolutions, and/or indifferent packet/error correction structures. In some cases, somecompression resources can be shared amongst multiple compressionprocesses compressing the same video/audio (e.g., in many compressionalgorithms, there is a step whereby the image is scaled to multiplesizes before applying compression. If different size images are requiredto be output, then this step can be used to serve several compressionprocesses at once). In other cases, separate compression resources willbe required for each format. In any case, the compressed video/audio 339of all of the various resolutions and formats required for a givenapp/game server 321-322, 325 (be it one or many) will be output at onceto outbound routing 340. In one embodiment the output of the compressedvideo/audio 339 is in UDP format, so it is a unidirectional stream ofpackets.

The outbound routing network 340 comprises a series of routing serversand switches which direct each compressed video/audio stream to theintended user(s) or other destinations through outbound Internet traffic399 interface (which typically would connect to a fiber interface to theInternet) and/or back to a delay buffer 315 (which may be implementedusing one of the RAID arrays), and/or directly back to the inboundrouting 302, and/or out through a private network (not shown) for videodistribution. Note that (as described below) the outbound routing 340may output a given video/audio stream to multiple destinations at once.In one embodiment this is implemented using Internet Protocol (IP)multicast in which a given UDP stream intended to be streamed tomultiple destinations at once is broadcasted, and the broadcast isrepeated by the routing servers and switches in the outbound routing340. The multiple destinations of the broadcast may be to multipleusers' clients 215 via the Internet, to multiple app/game servers321-322, 325 via inbound routing 302, and/or to one or more delaybuffers 315. Thus, the output of a given server 321-322 is compressedinto one or multiple formats, and each compressed stream is directed toone or multiple destinations.

Further, in another embodiment, if multiple app/game servers 321-322,325 are used simultaneously by one user (e.g., in a parallel processingconfiguration to create the 3D output of a complex scene) and eachserver is producing part of the resulting image, the video output ofmultiple servers 321-322, 325 can be combined by the shared hardwarecompression 330 into a combined frame, and from that point forward it ishandled as described above as if it came from a single app/game server321-322, 325.

Note that in one embodiment, a copy (in at least the resolution orhigher of video viewed by the user) of all video generated by app/gameservers 321-322, 325 is recorded in delay buffer 315 for at least somenumber of minutes (15 minutes in one embodiment). This allows each userto “rewind” the video from each session in order to review previous workor exploits (in the case of a game). Thus, in one embodiment, eachcompressed video/audio output 339 stream being routed to a user client215 is also being multicasted to a delay buffer 315. When thevideo/audio is stored on a delay buffer 315, a directory on the delaybuffer 315 provides a cross reference between the network address of theapp/game server 321-322, 325 that is the source of the delayedvideo/audio and the location on the delay buffer 315 where the delayedvideo/audio can be found.

App/game servers 321-322, 325 may not only be used for running a givenapplication or video game for a user, but they may also be used forcreating the user interface applications for the hosting service 210that supports navigation through hosting service 210 and other features.

Virtualization and Encryption System and Method for Hosting Applications

In one embodiment of the invention, the application/game servers321-322, 325 illustrated in FIG. 3 are implemented as virtualizedWindows platforms. Given that the application hosting service 210 mayhave thousands of application/game servers 321-322, 325 running Windowsand tens of thousands of users, it would be impractical to install everyapplication in the service to all Windows machines and replicateuser-specific changes for those applications across all these machines.One embodiment of the invention addresses these issues in a compatibleand scalable fashion using the virtualization techniques describedbelow. In addition, to ensure that the application program code remainssecure, one embodiment of the invention employs a variety of encryptiontechniques which will be discussed below in detail.

As illustrated in FIG. 4, one embodiment of the invention includes avirtualization and encryption service 412 to generate a virtualizedoperating system environment for the applications/video games 401executed on the various application/game servers. In the embodimentsdescribed herein, the virtualization and encryption service 412 isimplemented as a file system mini-filter driver for Windows. It shouldbe noted, however, that the underlying principles of the invention arenot limited to a Windows implementation.

As illustrated, the virtualization and encryption service 412 includes aregistry redirection module 402 for processing registry operations onbehalf of the applications/video games 401; a file redirection module403 for intercepting file system operations generated by theapplication/video games 401, directing those file operations to avirtual location when appropriate, and passing the modified operationsthrough the native file system 420 (which in one embodiment is the NTFSfile system used in current Windows environments); a decryption module404 for performing the various encryption and decryption techniquesdescribed herein; and a process and thread tracker module 405 forgenerating a hierarchical map of known allowable processes/threads andmonitoring active system processes and threads using the process map.Each of these system components is described in detail below.

As illustrated in FIG. 4, the various virtualization and encryptionservice components 402-405 interface with corresponding OS kernelcomponents 407-410, respectively, to perform their designated functions.In one embodiment the OS kernel components are Windows OS kernelcomponents; however, the underlying principles of the invention are notlimited to a Windows OS implementation. In particular, the registryredirection module 402 interfaces with a configuration manager 407 inthe OS kernel in response to registry operations. In one embodiment, theregistry redirection module 402 redirects all registry modifications toan in-memory representation of the registry 407, sometimes referred toas a “registry hive.” For example, upon detecting that an application orvideo game has generated a registry read/write operation to a particularregistry key, the registry redirection module 402 intercepts and pendsthe operation, generates a new request directed at the registry hive onbehalf of the application/video game, copies the results from therequest, and completes the pending operation. In one embodiment,registry hive data 432, application program files 433, and file metadata431 are stored within a container 434 data structure which (as describedbelow) is secure and easily ported between application/game servers.When the active containers 434 are unloaded from memory (e.g., when theuser stops playing a video game and logs off of the system) the registrykey modifications made to the in-memory registry metadata 413 arepersistently stored as registry hive data 432 within a container 434associated with the user.

Subsequently, when the user logs back in to the system (e.g., to playthe same video game or execute the same application 401), the registrychanges are read from the stored registry hive data 432 and loaded intomemory. Any registry operations to these registry keys are thenintercepted by the registry redirection module 402 and redirected to thein-memory representation 413.

Additionally, in response to the file system operations generated by thecurrently-executing application/video game 401, a file redirectionmodule 403 redirects the file system operations to the container 434which stores virtualized copies of the files 433 and file metadata 431.By way of example, when an application is initially installed on acomputer system, a setup program (e.g., “setup.exe”) installsapplication files to a specific location on a local hard drive such as“C:/program files/EA/Crysis/.” These files are represented in FIG. 4 asunmodified application files 435 which, in one embodiment, aredesignated by the virtualization and encryption service 412 as “readonly.” In one embodiment, in response to a subsequent request by anapplication 401 to modify the files, the file redirection module 403modifies the path to the files for these operations to a virtual filelocation 433 associated with the container 434 such as “D:/programfiles/EA/Crysis/.” Using the modified path, the files 433 are accessiblevia the native file system 420 (which in a Windows implementation istypically a New Technology File System (NTFS)). In one embodiment, thevirtual location of the files for each user and machine is identified byfile metadata 407 stored in memory as the application 401 is executed.The registry and file metadata 407 is updated to track each registry andfile operation. When the application is closed (e.g., when the user logsoff the system), the file metadata 431 is stored within the samecontainer 434 as the underlying files 433 so that it may be used insubsequent instantiations of the application. In this manner, a virtualfile system identified by the redirection module 403 is effectivelymerged into the real file system in which the unmodified files andregistry 435 are stored.

In one embodiment, multiple containers 434 are maintained on the systemfor each user and application. For example, for a particular videogame—the video game Crysis™ will be used as an example—differentcontainers 434 may be maintained which contain different versions of themetadata 431 and application program files 433. The different containersmay be identified using a logical naming convention (e.g., Crysis1.olc,Crysis2.olc, Crysis3.olc, etc). In one embodiment, all current user dataand other modified program files are stored within a designatedcontainer known as a user-data container and each user is assigned adifferent user-data containers per game. Multiple related applicationscan share a single user-data container for a user. Multiple users can bedesignated to share a single shared user-data for an application.Different containers may also be used for application updates (e.g.,patches created by the application publisher). When a particular userlogs in to the system to play a particular video game or use aparticular application, all containers related to a game/application anda specific user-data container will be loaded by the virtualization andencryption service 412. As previously mentioned, in one embodiment,there is a separate database system that tracks the location forapplication specific user-data containers.

As mentioned above, the same basic principles apply to registry updates.For example, in response to a request by an application to update theregistry key HKLM/Software/OnLive the registry redirection module 402will pend the request, perform the update to a virtual location with acopy of the registry key identified by the metadata 407 (e.g.,HKLM/Software/OnLive stored in any of the containers), get the resultsof the update (e.g., success and value or failure), and pass the resultsback to the pending request.

In one embodiment, when initially connecting to the hosting service 210,the user is designated an application game server 102 for the user'sapplication session. On a following connection to the hosting service210, the user may be designated a different application game server forthe same game. This requires the user-data generated from the lastsession to be localized within a server 102 for the new session. Thelocalization may include modifying the unique security ID(SID) for useraccounts, and accounting for differences in hardware and operatingsystems on the new server 102.

In one embodiment, the registry hive 432 illustrated in FIG. 4 does notcontain the entire registry but rather, only those registry keys whichhave been modified. As an application 401 writes to the registry, thewrite operations are redirected to the in-memory representation 413 ofthe registry keys. The registry changes are then stored persistentlywithin the registry hive data 432 of the container 434. When the usersubsequently logs back into the system, the modified registry keys aremerged with the complete registry data 435 (including unmodified keys).Registry read operations directed to previously-modified registry keysare then redirected to the in-memory representation 413.

In one embodiment, the virtualization and encryption service 412includes a process and thread tracking module 405 for tracking allactive processes/threads associated with an application 401. Inparticular, in one embodiment, when an application is initially loadedon the system, the process and thread tracking module 405 generates amap of the processes and threads and the hierarchy between the processesand threads which it stores as metadata. This includes, for eachprocess, all of its child processes and its parent and grandparentprocesses. In one embodiment, a process is identified by the full pathto the executable and the pre-computed SHA-256 hashing value.Subsequently, when an application is executed and attempts to load aseries of processes/threads into memory, the process and thread trackingmodule 405 reads the metadata via the metadata manager 406 and comparesthe map with the requested processes/threads. If a particular process orthread is not found in the map, or is found at a different level of theprocess/thread hierarchy than that stored in the map or doesn't matchthe hash or the command line arguments do not match, then the processand thread tracking module 405 may prevent the process/thread fromloading, may trigger an alert to be addressed by a system administrator,and/or may take automated corrective action. Generating a process/threadmap for each application and comparing the map to requestedprocesses/threads in this manner provides for additional security andmakes it difficult for a hacker to compromise the system using anunauthorized processes or threads.

In one embodiment, for the files associated with each application, someof the modified files 433 are encrypted for improved security. Inparticular, one embodiment of the virtualization and encryption service412 includes a decryption module 404 for encrypting/decryptingapplication files using a private/public key process which will now bedescribed.

In one embodiment, the online game hosting service 210 generates aRivest, Shamir and Adleman (RSA) public/private key pair and providesthe public key to each video game publisher. The publisher then uses aunique secret key to encrypt the application files (e.g., the video gameor application binaries), uses the public key to encrypt the secret key,and provides the encrypted application or video game files and encryptedsecret key to the application/video game hosting service 210. Theencrypted application/video game and encrypted secret key may then besecurely transmitted to the application/video game hosting service 210.The decryption module 404 may then use the private key of thepublic/private key pair to decrypt the secret key and then use thedecrypted secret key to decrypt the application/video game binaries. Inone embodiment, 256- or 512-bit Advanced Encryption Standard (AES)encryption is used. However, the underlying principles of the inventionare not limited to any particular type of encryption.

In one embodiment, the encrypted application/video game files are storedin the containers 434 in this encrypted format and the secret key isstored in a key store 440 managed by a key manager service 441 on a keyserver 436. Consequently, the containers 434 may be copied to any of theapplication/video game hosting servers 321-322, 325 securely. If amalicious user makes a copy of the files, they will be of no use to theuser without the secret key, which is stored securely within the keystore 440 under the control of the key manager service 406. In oneembodiment, the metadata 431 associated with the files includes anencryption attribute to indicate that the files in the container 434 areencrypted (i.e., so any write request to a file is completelyprotected).

As mentioned above, in one embodiment, the encrypted secret keys ofapplication publishers are stored within a secure key store 440 managedby a key manager service 441. These secret keys are never transmitted toapplication/game servers of the hosting service 321-322, 325 in anunencrypted state. In one embodiment, each instance of thevirtualization and encryption service 412 generates its own uniquepublic/private key pair and provides the public key to the key manager441. The key manager 441 then uses the public key to encrypt the secretkeys of application/game publishers when transmitting the secret keys todifferent instances of the virtualization and encryption service 412.The decryption module 404 of each instance of the virtualization andencryption service 412 may then use the private key to decrypt theapplication files and other encrypted data 432-433 stored within thecontainer 434 for a particular user and application.

In one embodiment, authentication techniques are employed to ensure thatonly authorized processes and threads can access the encrypted content.The process and thread tracker 405 maintains a list of trusted processesand threads who are allowed to view encrypted content. Consequently,upon receipt of a request by a particular process to access theencrypted content, the process and thread tracker module 405 may comparethe instantiation of the process/thread to the trusted list to ensurethat the process/thread is authorized to execute and access theencrypted content.

In one embodiment, the authentication techniques are employed to ensurethat only authorized processes and threads can execute in the system. Inparticular, as previously mentioned, the process and thread trackermodule 405 maintains a map of allowed processes and threads includingthe hierarchical relationships between the processes and threads. Inaddition, the process and thread tracker module maintains a full path tothe executable, command-line parameters and SHA-256 hashing. Uponprocess or thread creation, the process and thread tracker moduleconfirms that this is an allowed process or thread by verifying itshierarchical relationships, hashing value and command-line parameters.If process creation doesn't match expected values then creation isblocked.

In one embodiment, the file system operations redirected by the fileredirection module 403 and the encryption of the application/video gamebinaries occurs transparently to the executing applications/video games(i.e., the applications are unaware that the underlying files areencrypted). When cypher-block chaining is used to encrypt a file asdescribed above, the encrypted file may be larger in size than theunencrypted file (i.e., to account for the encryption overhead).Consequently, in one embodiment, when reading or writing a file, thevirtualization and encryption service 412 initially opens a larger,virtual file system buffer than is typically used for reading/writingthe unencrypted file. For example, if an application needs 30unencrypted bytes from a file, the NTFS file system would normallyallocate a 30-byte buffer. In one embodiment of the invention, however,the file redirection module 403 may need to swap out the 30-byte bufferfor a 48-byte virtual buffer to read 48 bytes in order to properlydecrypt the file. The 48 bytes may then be read into the virtual bufferand decrypted to generate the 30 byte file. The file redirection module403 may then swap out the 48 byte buffer for a 30 byte buffer containingthe decrypted content and provide the 30 byte buffer to the requestingapplication 401. In one embodiment, the read buffers are implemented asstandard NTFS target buffers that the NTFS file system 420 fills withdata. As previously mentioned, however, that the underlying principlesof the invention are not limited to an implementation using the NTFSfile system.

In one embodiment, before a container is loaded, one embodiment of thevirtualization and encryption engine 412 ensures that there are no openhandles to a requested file in the virtual file system (to preventmalicious users from breaching the system using open handles).Similarly, when a container is unloaded (e.g., in response to a userlogging out of an application), all of the in-memory container file'sdata 407 is purged so that all the decrypted contents are flushed out ofsystem memory. In addition, in one embodiment, the virtualization andencryption service 412 ensures that there are no open handles toencrypted files when a container is unloaded. In one embodiment, if ahandle is detected then the system is rebooted so that all processeswhich have access to these open handles are terminated.

In one embodiment, because the virtualization and encryption service 412tracks multiple containers, files may be spread out to decrease loadtimes. Additionally, because the containers are portable for the reasonsset forth above, they may be stored on different machines or sharednetwork drives. Typically, an application loads code and data sectionsin bursts separated by long periods of execution times. Thus, acontainer may be carved out into multiple containers and spread out sothat data that is accessed early on is kept closer to the machine thatexecutes these applications/video games. Consequently, load anddistribution times may be decreased without consuming disk space. Thisallows us to do away with having copies of all applications on each ofthe machines that execute them.

One embodiment of a method for installing an application in a hostedvirtual environment is illustrated in FIG. 5. The method may beimplemented within the context of the system architecture shown in FIG.4. However, the method is not limited to any particular systemarchitecture.

At 501, the application installation is initialized. For mostapplications, this means executing a “setup.exe” program which copiesapplication binaries to a designated folder on the hard drive andupdates the system registry. At 502 a, installation commands to copybinaries are detected and, in one embodiment, are redirected to adesignated virtual path at 502 c. As previously described, if the pathis, for example, C:/program files/Crysis705, then the installation pathmay be redirected to D:/program files/Crysis705. In one embodiment, theupdates are made to read-write container identified by the path (asdescribed above). In a similar manner, at 502 b, installation commandswrite to the registry. The game capture in a read-write containerbecomes a read-only container during game execution. During this time,the read-write container is the user-data for that game.

At 502 d, registry updates made by the installation program are detectedand translated to a virtual registry hive. For example, in response to arequest by an application to update the registry keyHKLM/Software/OnLive stored in the hive on the C: drive, the registryredirection module 402 will pend the request, perform the update to avirtual location with a copy of the registry key identified by themetadata 407 (e.g., HKLM/Software/OnLive stored on the D: drive), getthe results of the update (e.g., success and value or failure), and passthe results back to the pending request.

At 503, metadata identifying the new locations for the files andregistry keys is updated. Subsequently, when the user logs out of theapplication, the metadata, files and registry updates are stored withina container (as previously described).

One embodiment of a method for executing an application and recordingfile updates in a hosted virtual environment is illustrated in FIG. 6.The method may be implemented within the context of the systemarchitecture shown in FIG. 4. However, the method is not limited to anyparticular system architecture.

At 601, the metadata for all application containers is loaded in memoryand localized and, at 602, the application is executed. At 603, a filesystem operation by the application is detected and, at 604, adetermination is made as to whether the file system operation isdirected to a real/physical, virtualized, or merged (combinedreal/physical and virtual) location (e.g., to C:/program files/programname). If real, the operations are passed through to the base filesystem at 605; if virtual, then at 607, the path is translated to thevirtual location (e.g., D:/program files/program name) and the filesystem operation is executed; if merged, then at 606, the results arecombined from the base file system and the virtual location. Metadata isupdated as necessary at 608 and (if necessary to identify the updates tothe virtual location). At 609, the results of the operation are providedto the application and, at 610, the application exits (e.g., in responseto the user logging out of the system). At 611, the updated metadata isstored within the application container (as described above).

One embodiment of a method for executing an application and recordingregistry updates in a hosted virtual environment is illustrated in FIG.7. The method may be implemented within the context of the systemarchitecture shown in FIG. 4. However, the method is not limited to anyparticular system architecture.

At 701 a, the application container with the metadata and registry hiveis loaded into memory. The registry hive is imported from a registryhive file and mounted onto a virtualized location in memory for theactive registry file. The registry keys may be stored in a standardformat and imported from a .reg file and/or mounted from a .dat file.The underlying principles of the invention remain the same regardless ofhow the registry keys are imported/mounted.

At 701 a a determination is made as to whether the current user SIDmatches the SID indicated in the registry. As mentioned above, a useraccount on a particular machine has a unique SID associated with it(identified by a key such asHKEY_USERS\S-1-5-21-2118976691-607243524-348483912-1295 andHKEY_USERS\S-1-5-21-2118976691-607243524-348483912-1295_Classes).Consequently, when a user logs in and out from one application/gameserver (e.g., server 321) and subsequently logs in to a differentapplication/game server (e.g., server 322), the same user account willhave a different SID on the new machine. To address this issue, in oneembodiment of the invention, before the registry key is mounted for aparticular registry hive 432, an SID match is performed at 701 a. If thecurrent SID does not match the SID stored in the registry hive then, at701 a, an SID resolution algorithm is executed to update the SID valueassociated with the current registry hive. Once the SID is fixed to thecorrect value, the registry hive is loaded at the appropriate mountpoint and the user is permitted to execute the application.

At 705, registry operations are detected as the application executesand, at 706, the registry updates are redirected on behalf of theapplication to the in-memory registry hive. The results of the updates(e.g., successful and value or failure) are provided to the application.At 707 a, the metadata associated with the registry is updated toreflect the updates and at 707 b the results are provided to theapplication. At 708, the application is terminated (e.g., in response tothe user logging out of the application) and, at 709, the registry keymodifications and metadata are persistently stored within theuser/application container.

One embodiment of a method for encrypting application program code isillustrated in FIG. 8. The method may be implemented within the contextof the system architecture shown in FIG. 4. However, the method is notlimited to any particular system architecture.

At 801, a public/private key pair associated with the applicationhosting service is generated and the public key is provided toapplication publishers. At 802, a secret key is generated for aparticular publisher (Key X for Publisher Y). At 803, the applicationfiles for a particular application or video game are encrypted using thesecret key, and the secret key is encrypted using the public key. At804, the encrypted secret key is stored within a secure key store (e.g.,such as the key store 440 shown in FIG. 4) and the encrypted applicationfiles are stored within a database or file system on the applicationhosting service.

One embodiment of a method for securely decrypting and executingapplication program code generated by an application publisher isillustrated in FIG. 9. The method may be implemented within the contextof the system architecture shown in FIG. 4. However, the method is notlimited to any particular system architecture.

At 901, each application/game server 321-322, 325 generates its ownunique public/private key pair. At 902, the machine-specific public keyis provided to the key server (e.g., key server 436 in FIG. 4). At 903,the key server decrypts the secret key (previously encrypted by theapplication publisher along with the application binaries) using theprivate key of the hosting service. At 904 the key service re-encryptsthe secret key using the machine-specific public key (generated in step901) and, at 905, transmits the encrypted secret key to theapplication/game server 321-322, 325 on which the application is to beexecuted. At 906, the machine decrypts the encrypted secret key with themachine-specific private key. Finally, at 907, the application/videogame hosting server decodes the container files 431-433 using thedecrypted secret key and the application is executed. Thus, using thesetechniques, the binaries, registry data and other metadata for videogames and other types of applications may be encrypted and securelytransmitted between a storage location (e.g., RAID Arrays 405 shown inFIG. 1) and an application/game server (e.g., servers 321-322, 325 inFIG. 3).

One embodiment of a method for securely executing application programcode on a hosting service is illustrated in FIG. 10. The method may beimplemented within the context of the system architecture shown in FIG.4. However, the method is not limited to any particular systemarchitecture.

At 1001, application program code is installed on a hosting service(e.g., by running a setup.exe or similar executable program). At 1002,the application is executed and analyzed to generate a process/threadmap which identifies the processes/threads of the executing applicationand the relationships between the processes/threads. For example, asmentioned above, the process/thread map may indicate, for each process,all of the child processes, parent processes, and grandparent processes.The process/thread map is then stored in persistent storage for lateruse.

At 1003, a request is received to execute the application program code(e.g., a user request to execute a particular video game). At 1004, theprocesses/threads to be executed are compared against the process/threadmap to ensure that all processes/threads to be executed are found in themap at the correct hierarchical location (i.e., with the same sets ofchild, parent, and grandparent processes). If any requestedprocess/thread, or it's hierarchical position does not match, determinedat 1005, at 1006 a decision is made to not execute the applicationand/or other precautionary measures may be taken (e.g., such asrebooting the machine and/or notifying a system administrator). Asmentioned, each process may be identified by its process ID. If,however, each process/thread has a matching process thread in the map atthe correct hierarchical location, then at 1007, the processes/threadsof the application are allowed to execute.

An exemplary container format used for packaging files (e.g., video gamefiles) into file containers 434 in one embodiment of the invention isillustrated in FIG. 11. In one embodiment, the illustrated containerformat may be used as a general replacement for Zip or tar to holdapplication files, video assets, metadata, and other system informationdescribed herein.

As illustrated, the container format of this embodiment comprises aheader portion 1101, a file portion 1102 and a directory portion 1103.The header portion contains various different types of metadata relatedto container including, by way of example and not limitation, theversion of the container files, flags, a directory position indicatingthe location of the directory portion 1103 within the container, adirectory length indicating the size of the directory portion 1104, adirectory checksum containing a checksum on the directory portion 1104,a pub len and app len, a publisher name indicating the name of thepublisher of the container, a game name indicating the name of the gamepackaged in the container and a header checksum containing a checksum onthe header portion 1101.

The file portion 1102 contains the actual files of the container and thedirectory portion 1103 contains information related to each of the filesincluding the file position within the container (e.g., as an offsetvalue), file length indicating the size of each file, file flagsindicating, name length indicating the length of the name for each file,a file checksum containing a checksum over each file, and a file namefor each file.

The features provided by the file container format includepublisher-controlled file alignment within the container; automated filere-ordering to minimize seeking; and the ability to distribute pieces ofthe container across multiple servers (e.g., preventing the entire gamefrom being in a single location).

One embodiment of the invention supports the ability to add and removefeatures for an application without re-starting an application. Forinstance, a game may provide a weaponry selection as an add-on feature.Once a user purchases a weapon, only his game has access to thoseweapons. Instead of compiling separate binaries for each combination ofweapons, these features may be executed dynamically as add-on features.In one embodiment, the add-on features are implemented as specialcontainers that can be loaded/unloaded while the game is running (withparticipation from the games).

In one embodiment, the various functional modules illustrated herein andthe associated steps may be performed by specific hardware componentsthat contain hardwired logic for performing the steps, such as anapplication-specific integrated circuit (“ASIC”) or by any combinationof programmed computer components and custom hardware components.

In one embodiment, the modules may be implemented on a programmabledigital signal processor (“DSP”) such as a Texas Instruments' TMS320xarchitecture (e.g., a TMS320C6000, TMS320C5000, . . . etc). Variousdifferent DSPs may be used while still complying with these underlyingprinciples.

Embodiments may include various steps as set forth above. The steps maybe embodied in machine-executable instructions which cause ageneral-purpose or special-purpose processor to perform certain steps.Various elements which are not relevant to these underlying principlessuch as computer memory, hard drive, input devices, have been left outof some or all of the figures to avoid obscuring the pertinent aspects.

Elements of the disclosed subject matter may also be provided as amachine-readable medium for storing the machine-executable instructions.The machine-readable medium may include, but is not limited to, flashmemory, optical disks, CD-ROMs, DVD ROMs, RAMs, EPROMs, EEPROMs,magnetic or optical cards, propagation media or other type ofmachine-readable media suitable for storing electronic instructions. Forexample, the present invention may be downloaded as a computer programwhich may be transferred from a remote computer (e.g., a server) to arequesting computer (e.g., a client) by way of data signals embodied ina carrier wave or other propagation medium via a communication link(e.g., a modem or network connection).

It should also be understood that elements of the disclosed subjectmatter may also be provided as a computer program product which mayinclude a machine-readable medium having stored thereon instructionswhich may be used to program a computer (e.g., a processor or otherelectronic device) to perform a sequence of operations. Alternatively,the operations may be performed by a combination of hardware andsoftware. The machine-readable medium may include, but is not limitedto, floppy diskettes, optical disks, CD-ROMs, and magneto-optical disks,ROMs, RAMs, EPROMs, EEPROMs, magnet or optical cards, propagation mediaor other type of media/machine-readable medium suitable for storingelectronic instructions. For example, elements of the disclosed subjectmatter may be downloaded as a computer program product, wherein theprogram may be transferred from a remote computer or electronic deviceto a requesting process by way of data signals embodied in a carrierwave or other propagation medium via a communication link (e.g., a modemor network connection).

Additionally, although the disclosed subject matter has been describedin conjunction with specific embodiments, numerous modifications andalterations are well within the scope of the present disclosure.Accordingly, the specification and drawings are to be regarded in anillustrative rather than a restrictive sense.

1. A computer-implemented system for providing a virtualized applicationhosting environment comprising: a plurality of application/game serverseach having a memory for storing application program code and data and aprocessor for executing the application program code and processing thedata on behalf of a client; a virtualization service executed on each ofthe application/game servers, the virtualization service comprising afile redirection module for redirecting file system operations generatedby the application program code to the base file system or one or morevirtualized file locations; and a registry redirection module forintercepting registry operations generated by the application programcode and executing the registry operations on behalf of the applicationprogram code at the base or one or more virtualized registry locations.2. The system as in claim 1 wherein redirecting the file systemoperations further comprises modifying a path associated with the filesystem operations to a virtualized path identifying a virtualizedlocation in a memory and/or storage device.
 3. The system as in claim 1wherein intercepting the registry operations further comprises: pendingthe registry operation; executing the registry operation in thevirtualized registry location on behalf of the application program code;and providing the results back to the pended registry operation.
 4. Thesystem as in claim 3 wherein the results comprise an indication of asuccessful registry update and an associated value or an indication of aregistry update failure.
 5. The system as in claim 1 wherein theapplication program code comprises an online video game and wherein theapplication/game servers execute the online video game and responsivelygenerate an audio/video output signal in response to video game controlcommands received from the client.
 6. The system as in claim 5 furthercomprising: an audio/video compression module for compressing theaudio/video output signal from the online video game using a low latencyaudio/video compression algorithm.
 7. The system as in claim 6 whereinthe audio/video compression module compresses separate independentaudio/video streams for each of a plurality of different clients.
 8. Thesystem as in claim 1 wherein the application program code and data isstored within an encrypted application container specific to a user ofthe client, wherein the encrypted application container is read intomemory and decrypted prior to execution of the application program code.9. The system as in claim 8 wherein a separate encrypted applicationcontainer is maintained for each individual user and application. 10.The system as in claim 8 further comprising: a metadata manager forstoring metadata associated with each of the file system operationsredirected to the one or more virtualized file locations and registryoperations executed on behalf of the application program code at one ormore virtualized registry locations.
 11. The system as in claim 10wherein when a user of the client logs off the system, the metadata,file and registry updates are stored within the application containerassociated with the user.
 12. A computer-implemented method forproviding a virtualized application hosting environment comprising:executing application program code on an application/game server onbehalf of a client; redirecting file system operations generated by theapplication program code to one or more virtualized file locations; andintercepting registry operations generated by the application programcode and executing the registry operations on behalf of the applicationprogram code at one or more virtualized registry locations.
 13. Thesystem as in claim 12 wherein redirecting the file system operationsfurther comprises modifying a path associated with the file systemoperations to a virtualized path identifying a virtualized location in amemory and/or storage device.
 14. The system as in claim 12 whereinintercepting the registry operations further comprises: pending theregistry operation; executing the registry operation in the virtualizedregistry location on behalf of the application program code; andproviding the results back to the pended registry operation.
 15. Thesystem as in claim 14 wherein the results comprise an indication of asuccessful registry update and an associated value or an indication of aregistry update failure.
 16. The system as in claim 12 wherein theapplication program code comprises an online video game and wherein theapplication/game servers execute the online video game and responsivelygenerate an audio/video output signal in response to video game controlcommands received from the client.
 17. The system as in claim 16 furthercomprising: an audio/video compression module for compressing theaudio/video output signal from the online video game using a low latencyaudio/video compression algorithm.
 18. The system as in claim 17 whereinthe audio/video compression module compresses separate independentaudio/video streams for each of a plurality of different clients. 19.The system as in claim 12 wherein the application program code and datais stored within an encrypted application container specific to a userof the client, wherein the encrypted application container is read intomemory and decrypted prior to execution of the application program code.20. The system as in claim 19 wherein a separate encrypted applicationcontainer is maintained for each individual user and application. 21.The system as in claim 19 further comprising: storing metadataassociated with each of the file system operations redirected to the oneor more virtualized file locations and registry operations executed onbehalf of the application program code at one or more virtualizedregistry locations.
 22. The system as in claim 21 wherein when a user ofthe client logs off the system, the metadata, file and registry updatesare stored within the application container associated with the user.